Effects of strategy-migration direction and noise in the evolutionary spatial prisoner's 

dilemma 
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Spatial games are crucial for understanding patterns of cooperation in nature (and to some extent 
society). They are known to be more sensitive to local symmetries than e.g. spin models. This paper 
concerns the evolution of the prisoner's dilemma game on regular lattices with three different types 
of neighborhoods — the von Neumann-, Moore-, and kagome types. We investigate two kinds 
of dynamics for the players to update their strategies (that can be unconditional cooperator or 
defector). Depending on the payoff difference, an individual can adopt the strategy of a random 
neighbor (a voter-model-like dynamics, VMLD), or impose its strategy on a random neighbor, i.e., 
invasion- like dynamics (IPLD). In particular, we focus on the effects of noise, in combination with 
the strategy dynamics, on the evolution of cooperation. We find that VMLD, compared to IPLD, 
better supports the spreading and sustaining of cooperation. We see that noise has nontrivial effects 
on the evolution of cooperation: maximum cooperation density can be realized either at a medium 
noise level, in the limit of zero noise, or in both these regions. The temptation to defect and the local 
interaction structure determine the outcome. Especially, in the low noise limit, the local interaction 
plays a crucial role in determining the fate of cooperators. We elucidate these both by numerical 
simulations and mean-field cluster approximation methods. 

PACS numbers: 89.75.Fb, 87.23.Kg, 87.23. Ge, 05.50.+q 



I. INTRODUCTION 

In the last decades, a great deal of interest has been de- 
voted to understanding the evolution of cooperation [l[ , 
i.e., how cooperation emerges and persists in a popula- 
tion composed of selfish individuals. Evolutionary game 
theory 0, [| - a research field cultivated tradition- 
ally by both evolutionary biologists, political scientists, 
economists and mathematician 0, H| that recently has 
attracted a growing interest from the physics commu- 
nity @, 0] — provides a general framework to address 
such issues. One famous metaphor for the problem of co- 
operation is the prisoner's dilemma (PD) game [1, Q, in 
which defection could benefit the individual in the short 
term, whereas cooperation has only potential benefits in 
a longer time perspective. In the common mathemati- 
cal formulation of PD with pairwise interaction, each of 
two encountering cooperators (defectors) get a payoff R 
(P), a defector confronting a cooperator acquires payoff 
T, while the cooperator gains S. The four parameters is 
required to satisfy the conditions T > R > P > S and 
2R > T + S. In a well-mixed population, defectors are 
unbeatable and cooperators are doomed to extinction [J] . 

The seminal work of Nowak and May [lfj, where a 
PD game is played on two-dimensional grids with agents 
lacking memory and ability to form complex strategies, 
showed that spatial structure and nearest neighbor inter- 
actions can enable cooperators to survive by forming clus- 
ters within which they benefit from mutual cooperation 
and protecting them from exploitation by defectors. This 



* zhi-xi.wu@physics.umu.se 
tpetter.holme@physics.umu.se 



work has inspired numerous subsequent investigations of 
evolutionary games on spatial grids [HE]- Now the spa- 
tial (or network) reciprocity is regarded as one of the five 
main mechanisms supporting cooperation 0, [Til ]. Fur- 
thermore, coevolution of strategy distribution and under- 
lying interaction grap hs can also be useful in characteriz- 
ing the PD game jl5l [l6l. UtI [l8j . The crucial observation 
is that through switching partners, cooperators are capa- 
ble of frequently meeting other cooperators, which sub- 
stantially increase their reproduction rate. In addition, a 
lots of other mechanisms favoring cooperation have also 
been proposed: volunteering participation separa- 
tion of interaction and learning graphs |20l , dilution and 
random diffusion of agents on the grid [2l| , success-driven 
migration 12211 . memory effects [23[, reputation-based in- 
teraction [24| . moderate aspiration level [25|, etc. For a 
comprehensive review of this research field, we refer to 
Refs. HQ. 

It is worth pointing out that the scale-free topology, 
thanks to the capability of mutual protection of the hubs 
if occupied by cooperators, can enhance the spreading of 
cooperative behavior and resist the invasion of defection, 
hence becoming a promoter of cooperation fl3j . How- 
ever, it was also argued that the adoption of averaged 
payoffs instead of accumulated ones in the score func- 
tion, as well as an introduction of participation costs, 
might weaken this mechanism [2(| . These results suggest 
that the distinct ability of strategy breeding of the play- 
ers [27j , rather than the interaction graph, matters much 
for the emergence of cooperation. The heterogeneous 
or asymmetric strategy migration ability can be imple- 
mented in different ways, such as dynamic preferential 
selection (28j , introduction of two types of players with 
different teaching activity 2£ | , social diversity [3(| E[ , 
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long as some distinguished players have higher influence 
to spread their strategy [31(, and also the connections 
among these influential players are coupled in some ap- 
propriate way (27l . [29l . [30| , the dilemma can be relaxed 
and the promotion of cooperation is warranted. 

These results suggest that the heterogeneity in the mi- 
gration of strategies might be an important factor for 
the stability of cooperation. To our knowledge, in most 
previous studies of spatial games, a common simplifying 
assumption is that, whenever updating strategy, a player 
selects a neighbor as reference and attempts to adopt 
the neighbor's strategy according to some prescribed cri- 
terion. In other words, the focal player is always a recip- 
ient of the strategy. Only few works have studied the re- 
verse situation, with strategy donors and their influence 
on the evolution of cooperation. Ohtsuki et al. imple- 
mented both death-birth and birth-death updating for 
games on graphs, and found that for birth-death updat- 
ing, selection does not favor cooperation [34| . Antal et al. 
studied evolutionary dynamics on degree-heterogeneous 
graphs, evolved either by one individual dying and being 
replaced by the offspring of a random neighbor (voter 
model dynamics) or by an individual giving birth to an 
offspring taking over a random neighbor (invasion pro- 
cess dynamics) . They found that the fixation probability 
of a single fitter mutant under the voter model dynamics 
is k 2 times of corresponding value for the invasion process 
dynamics (k is the node degree) (35|. With this paper, 
we continue investigating the effects of the direction of 
strategy migration on the evolution of cooperation in the 
spatially explicit PD game. Another theme of recent re- 
search is the effects of noise on dynamic processes [36j . 
Indeed, noise (or mutation) plays an important role in 
the outcome of evolutionary games [H HI . In recent 
years, the effects of noise on spatial games have also been 
studied Sffl El 51 El ■ The spatial PD model pro- 
posed in [k| is a deterministic cellular automaton and 
can be extended to a stochastic version by introduction 
of noise in different ways. Perhaps the first spatial PD 
model with a stochastic strategy adoption process was 
proposed in Ref. (44|. We take the approach of Szabo 
and Toke [39j , where the players updates their strategies 
according to a Fermi function and in which noise have the 
role of temperature in the kinetic Ising model. For this 
simple "noise-guided" evolutionary model, the authors 
found rich dynamic phenomena — two absorbing states 
consisting of only cooperators and defectors separated 
by a coexistence region. Particularly, the phase transi- 
tion involving the extinction of cooperators or defectors is 
found to belong to the universality class of directed per- 
colation [H . In a similar vein, they also studied how the 
noise affects the phase diagram of PD on different two- 
dimensional lattices and Newman- Watts networks [40| . 
In somewhat different approach, Perc et al. and Tanimoto 
introduced noise to the payoff matrix El El ■ The re- 
ported results show a coherence-resonance phenomenon 
where the fraction of cooperators reaches its maximum 
at an intermediate noise level [431. In Ref. [401 the noise 



effect in PD games was studied, mainly focusing on the 
threshold of extinction of cooperation as a function of 
noise intensity. A clear picture for how the average frac- 
tion of cooperation evolves by varying the noise level un- 
der different intensities of the temptation to defection, to 
the best of our knowledge, is still lacking. In the present 
work, we will follow the research line of Refs. [34l. l35l. El 
to study how the strategy migration direction and se- 
lection noise affect the cooperation. In the rest of the 
paper, we first define our model and then treat the prob- 
lems sketched above both by computer simulations and 
cluster approximation methods. Finally, we make some 
discussions and draw our conclusion. 



II. MODEL 

In the present study, we consider the evolutionary PD 
on three types of regular lattices with periodic bound- 
ary conditions, namely square lattice with von Neumann 
neighborhood (von Neumann lattice), square lattice with 
Moore neighborhood (Moore lattice), and kagome lattice, 
such that the number of encounters of each player are 
four, eight, and six, respectively. The reason we concen- 
trate on these simple lattices is twofold: first, previous 
studies have found that local interaction does affect the 
spreading of cooperation [11 El; second, their regular 
structures allow us to implement mean-field cluster ap- 
proximation analysis 0, 0, El . 

Following many studies 10. l3ll. l39l . El, we use the 
Nowak-May parameterization pj| of the spatial PD, i.e., 
the temptation to defect T = b (where 1 < b < 2), the 
reward for mutual cooperation R = 1, the punishment of 
mutual defection P = 0, and the sucker's payoff 5 = 0. 
Initially, we let the players be either unconditional coop- 
erators (C) or defectors (D) with equal probability. The 
evolution of strategies is governed by random sequential 
updating. For each of the considered lattice topologies, 
we first let a randomly chosen player i reap its payoff 
Pi by playing the PD game with its nearest neighbors. 
Then, we select a random neighbor j of i, and let it ac- 
quire its payoff Pj by playing the game with its nearest 
neighbors. 

When updating strategy, the focal player i can be of 
the role of either strategy donor or strategy- recipient. If 
i is the strategy-recipient, we implement a voter-model- 
like dynamics ( VMLD ) 13511 . which is probably the most 
common approach @, (TllE [H, El EH, [H, S El EH, 
El, El- In this case, the player i imitates the strategy of 
7 with a probability dependent on their payoff difference 

m 
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l + exp [(Pi -!})/«]' 



(1) 



where re G [0, 00) denotes the noise level (or, in the lan- 
guage of evolutionary biology, selection intensity [3]). 
In the k = limit, the adoption of a successful strategy 
is deterministic, while in the re — ► 00 limit, the strategy 
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learning is blind [40(. If, however, we treat the player 
i as the strategy donor, we implement exactly invasion- 
process-like dynamics (IPLD) for the game [3a|. In this 
case, the neighbor j will try to take the focal player i's 
strategy with a probability defined as 



1 



l + expp^-POA 



(2) 



By tuning the values of b and k in the framework of the 
two updating schemes, we can obtain pictures of how the 
noise intensity and direction of strategy migration impact 
the final outcome of the evolutionary spatial PD game. 

In our simulation, we consider the possible strategy- 
migration for the players one by one according to a ran- 
dom sequence. One pass through all the agents is called 
a Monte Carlo (MC) sweep. Between each MC sweep we 
reshuffle the sampling sequence. The total population is 
of size 200 x 200 for the von Neumann and Moore lat- 
tices, and 3 x 100 x 100 for the kagome lattice. When 
repeating the above described elementary steps the sys- 
tem develops into a final stationary state characterized 
by the average fraction of cooperators f c , which is mea- 
sured for the last 3000 sweeps of the total 2 x 10 4 . All the 
simulation results presented below are averages over 20 
independent realizations of initial strategy distribution. 
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III. RESULTS AND DISCUSSION 

We start by comparing the results from the game with 
the VMLD strategy-updating rule with the correspond- 
ing IPLD results. In Fig. QJa), we present the simula- 
tion results for f c as a function of the temptation to 
defect b on the three lattices, and the noise level K is 
fixed as 0.3. We note that f c decreases monotonously 
with increasing & up to a threshold b c , where coopera- 
tion vanishes. Although the qualitative properties of the 
curves are somewhat similar, there exist quantitative dif- 
ferences. By comparison, the stationary cooperation level 
in IPLD is lower than the corresponding value for VMLD. 
Furthermore, IPLD results in a lower 6 c -threshold. Thus, 
our first finding is that VMLD is more favorable for the 
spreading of cooperation than IPLD. 

At a first glance, it may seem strange that VMLD and 
IPLD on homogeneous graph give rise to different re- 
sults, since what is regarded as VMLD from the point 
of view of the strategy-recipient is exactly the IPLD in 
the viewpoint of the strategy donor. (On heterogeneous 
graph, this difference is evident, since high-degree indi- 
viduals have greater chance to affect others for VMLD, 
and to bc affected by others for IPLD.) In fact, the four- 
or five-site cluster approximation, developed in the spirit 
of mean-field theory [6( , do not differentiate between the 
VMLD and IPLD, and give out exactly the same results 
for both strategy updating dynamics (see Fig. 1(b), the 
two curves for the VMLD and IPLD coincide with one an- 
other, and for simplicity we just show one). A crucial ob- 
servation is that this method neglects spatial correlations 



FIG. 1: (Color online) (a) Average fraction f c of coopera- 
tion as a function of the temptation to defect b for a fixed 
noise intensity k = 0.3 on three types of lattices. Solid and 
open symbols correspond to the two strategy-migration dy- 
namics VMLD and IPLD, respectively, (b) Theoretical es- 
timations by the four-site cluster approximation on the von 
Neumann lattice, and by the five-site cluster approximation 
on the kagome lattice, correctly predict the evolving tendency 
of f c , but do not differentiate between VMLD and IPLD dy- 
namics. Note the different scaling of the x-axis in the two 
panels. 



in the population, which cause the update probabilities 
of VMLD and IPLD to differ. Indeed, if the two prob- 
abilities are the same, the final results will be identical 
on homogeneous graphs. This argument can be deduced 
from the studies of voter model on graphs. In Ref. [47| . 
Castcllano studied both voter and anti-voter dynamics 
on networks (corresponding to the VMLD and IPLD in 
this work), and found that on homogeneous network, the 
consensus time in both dynamics is the same. In [35| . 
Antal et al. pointed out that the fixation probability of a 
neutral genotype on homogeneous graph is the same for 
both voter-model and invasion-process dynamics. The 
essential difference between the voter model and evolu- 
tionary games is that in voter model, when a randomly 
selected individual is to update its state, only the state of 
the selected neighbor matters, while in the PD game case, 
both the state of the selected neighbor and those of this 
neighbor's neighbors. For our setup, the average payoffs 
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FIG. 2: (Color online) Time series of Nc/Nd, the number of 
events of D switching to C divided by the reverse case, for 
b = 1.02 and 1.045 on the von Neumann lattice. 



collected by cooperators on the boundary is greater than 
that by defectors [Hj], we argue that VMLD favors the 
diffusion of cooperation on the rough boundaries sepa- 
rating cooperators from defectors stronger than IPLD. 
(In the Appendix, we have presented a concrete example 
to analyze the difference between the VMLD and IPLD 
determining the probability of strategy- migration.) A 
defector at a rough boundary would have, on average, 
greater chance to be convinced by its cooperating neigh- 
bors in the VMLD picture, than a cooperating neighbor 
to convince a defector in the IPLD. To test this point, 
we have traced the time series of Nc/Nd, the number of 
events of D switching to C divided by the reverse case, 
for two special values of b on the von Neumann lattice. 
The results presented in Fig. [5] give further evidence for 
the above speculation. 

Before moving forward, we briefly comment the im- 
portance of the network structures in maintaining co- 
operation. In previous studies [4fj| the overlapping tri- 
angles (especially, one-site overlapping triangles) in the 
connectivity structure are found to support the spreading 
(maintenance) of cooperation. Here, we want to point 
out that this conclusion is drawn in terms of the mag- 
nitude of b c for the evolutionary PD on different lat- 
tice structures (40j . Looking back on our simulations 
in Fig. [TJ we believe that this is not the whole story. 
If we measure the capability of promoting cooperation 
by the average fraction of cooperation in the equilibrium 
state, we observe that when b is small (e.g. b = 1.1), the 
one-site overlapping triangles (the kagome lattice) is an 
inferior structure compared to the case of multiple over- 
lapping sites (the Moore lattice). This indicates that the 
temptation parameter is an indicative variable when eval- 
uating the role of connectivity structure, giving a more 
nuanccd picture than the previously mentioned effect of 
overlapping triangles in Ref. • 
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FIG. 3: Average fraction f c of cooperation as a function of 
the noise parameter re for fixed values of b on (a) the von 
Neumann lattice, (b) the Moore lattice, and (c) the kagome 
lattice. The solid and open symbols are as shown in Fig. [I] 
The lines are guides to the eye. 



Now let us turn to the effect of noise. As mentioned 
above, the extinction threshold of cooperation b c as a 
function of re on different types of lattices has been stud- 
ied in [2^ . l40l | . Here we investigate how k influences f c in 
the stationary state for different values of b. In Fig.[3Ja), 
we show simulation results of f c versus re for two values of 
b = 1.02, b = 1.045 on the von Neumann lattice. We ob- 
serve so-called coherence resonance for both VMLD and 
IPLD, where f c reaches its maximum at an intermediate 




FIG. 4: (Color online) Replotting of the data in Fig.[3]for b = 
1.04 on the Moore lattice with the VMLD (a), and b = 1.12 
on the kagome lattice with the IPLD (b). The places where 
f c reaches its maximum value are indicated by arrows. 



FIG. 6: (Color online) Theoretical estimations for f c as a 
function of k by using the four-site cluster approximation 
method on the von Neumann lattice (a), the five-site cluster 
approximation on the kagome lattice (b). The arrows mark 
the places where f c maximizes when b = 1.12. 
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FIG. 5: Average fraction f c of cooperation versus k in the 
case of "high" temptation to defect b = 1.2 on the kagome 
lattice. The solid line denotes the analytical results obtained 
by the five-site cluster approximation. 



k. If, in this case, the noise is too weak or strong cooper- 
ation may be extinct (this is in accordance with previous 
results j43[). For the parameters used, the maximum 
f c emerges at about K = 0.3. We note that for the same 
temptation to defect, comparing with the case of VMLD, 
IPLD results in a narrower region of k, where coopcra- 
tors can maintain a non-zero fraction in the population. 
That is to say, cooperation is more robust against the 
fluctuation of noise in the case of VMLD. This result 
strengthens our previous finding from Fig. [TJ Note that 
limK^oo f c = 1/2. In this paper we focus on low noise 
levels, i.e. situations where the competitive interactions 
dominate the dynamics and do not study how coopera- 
tion reemerges for very large k. 

The observed resonant behavior can be understood as 
follows. As mentioned above, the noise parameter k mea- 
sures the stochastic uncertainties in the process of strat- 
egy learning. In the limit of weak noise, the irrational 
choices of the individuals become very rare. The strate- 
gies of those individuals who get higher payoffs would al- 
ways migrate successfully. Though the payoffs of the de- 
fectors at the interface are on average smaller than those 
of cooperators [33] , the highest payoff is always obtained 
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by those defectors surrounded by three other coopcra- 
tors. As such, if the individuals imitate with complete 
rationality (k — > 0), the defective strategy would spread 
more easily in the population, which does harm to the 
formation of clusters of cooperators. On the other hand, 
in the limit of large noise, the payoff information becomes 
less important in determining the success of strategy mi- 
gration. In such situation, it would be definitely ineffi- 
cient for the cooperators at the interface to spread their 
behavior (though they acquire higher payoff on average 
than those defectors [33[). If, however, the noise k is 
appropriately chosen, i.e., not too large or too small, the 
cooperators would have more chance to diffusion, and the 
defectors with highest payoff would have less probability 
to be followed by cooperators, leading to the surviving 
and enhancement of cooperation. Henceforth the opti- 
mal noisy intensity k emerges. 

The experimental tests of the PD game on the Moore 
and kagome lattices display somewhat similar results 
(Figs. Eljb) and (c)). For an appropriate temptation to 
defect, when varying «, we also notice an emergent of co- 
herence resonance. Moreover, the VMLD is also found to 
be superior to the IPLD in promoting cooperation. De- 
spite these two aspects, when n is sufficiently small, the 
simulation results illustrated in Fig.[3]Ja) are in stark con- 
trast to those in Figs. [3jb) and (c). In the former case, 
the cooperators arc doomed to extinction in the limit of 
zero noise, whereas they can persist in the population 
with finite f c in the latter two cases. We argue that the 
presence of overlapping triangles [4fj| in the latter two 
lattices contributes to the difference, since triangles are 
absent in the von Neumann lattice. Note, however, that 
the importance of overlapping triangles is evaluated with 
b c in [40| | , and with f c here. Comparing the curves for 
b = 1.04 in Figs.OHb) and (c), keeping Fig. Q] in mind, we 
conclude that for high temptation values, one-site over- 
lapping triangles gives more favorable conditions for co- 
operation, than the multiple-site case. For low tempta- 
tion values the situation is reversed. 

It is worth noting that the PD game on Moore and 
kagome lattices there is a nontrivial dependence of f c on 
k (which depends both on the temptation parameter and 
the direction of strategy migration). In particular, for 
sufficiently small b, we obtain only one maximum of f c as 
a function of k (the curve in Fig. [3{c) with b = 1.04 and 
VMLD). With increasing b, peculiarly, we observe two 
peaks of / c (k), one located at an intermediate value and 
the other at k = 0. This phenomenon is yet more promi- 
nent in the case of IPLD (c.f., Fig. 2]). Finally, if the value 
of b is sufficiently large (the curves of b = 1.1 in Fig. [3Jb) 
and also Fig. [5]), the peak of f c at finite k disappears and 
only the no-noise maximum remains. For comparison, in 
Fig. [5] we present analytic results via the four-site cluster 
approximation on the von Neumann lattice and five-site 
cluster approximation on the kagome lattice (for details, 
please see the Appendix in Ref. Q ) , and observe that the 
theoretical estimations correctly predict the tendency of 
the results obtained by our MC simulations. 
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FIG. 7: Average fraction f c of cooperation as a function of b 
in the case of very weak noise k = 0.01 on the kagome lattice 
(solid squares) and the Moore lattice (open circles). 



The simulation results shown in Figs. [31 [U and [5] allow 
us to speculate that in the limit of weak noise, the local 
interaction topology may play a decisive role in maintain- 
ing cooperation. To test this point further, we present 
in Fig. [7] the simulation results for the evolutionary PD 
on the Moore lattice and the kagome lattice, where f c as 
a function of b is plotted for a fixed value of k = 0.01. 
Here we only consider the VMLD. We find that the co- 
operators vanish at about b c = 1.5 for the kagome lattice 
(which has already obtained in Ref. [ibfl). and at about 
b c = 1.2 for the Moore lattice. Despite the value of b c , 
we note that the average fraction of cooperation f c seems 
to be insensitive to the temptation to defect staying as a 
plateau as cooperators are capable of surviving (an effect 
not studied in Ref. L 40| ) . In the following, we provide a 
local configuration analysis for f c in the stationary state 
on kagome lattices. 

In the limit of weak noise, the strategy-updating rules 
Eqs. (TT|) or 121) is equivalent to the deterministic imitation 
dynamics [T(|, i.e., as long as the payoff of the strategy 
donor is greater than that of the strategy recipient, the 
strategy will migrate. Since either an isolated coopera- 
tor, or a connected pair of cooperators, cannot prevail 
surrounded by defectors, consider a five-site cluster of 
cooperators fully surrounded by defectors. If b < 3/2, 
three cooperators forming a triangle will expand its ter- 
ritory to a five-site cluster, see Fig. [5] In that case we 
may expect that one of the neighboring defectors, say j, 
would since (b < 2) imitate i's strategy in the next sweep. 
Then their common neighbor k may acquire the payoff 
26 in the next sweep. However, since i gets a total payoff 
3; then, as long as b < 3/2, there is no chance that k's 
strategy can migrate to i. In other words, such a local 
configuration in a sea of defectors if b < 3/2 is stable. 
However, if a neighbor I of k was a cooperator, the total 
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FIG. 8: (Color online) Local configuration stability analysis 
in the limit of zero noise on the kagome lattice. The central 
five-site cluster of cooperators cannot be destroyed as long 
as b < 3/2 even if no neighbor or next-nearest neighbor is a 
cooperator. See the text for details. 



payoff of k would be 3& (which is greater than 3), and in 
that case fc's strategy could migrate to i, hence destroying 
the original cluster. Taken together, the necessary con- 
dition for a five-site cluster of cooperators existing stably 
on the kagome lattice requires that b < 3/2 and, at the 
same time, there is no cooperator in their next-nearest 
neighborhood. Since the second-nearest neighbors may 
be shared by other five-site cluster of cooperators, we 
can estimate f c to 5/(5 + 8 + 14/2) = 1/4 (8 and 14 are 
the number of nearest and next-nearest neighbors of the 
five-site cluster) . This estimation is very close to the one 
obtained by MC simulations 0.256(2). Thus this local- 
configuration stability analysis gives a good prediction for 
f c of evolutionary PD in the limit of weak noise on the 
kagome lattice. Indeed, we have visualized the networked 
population and found many isolated five-site clusters of 
cooperators on the kagome lattice (results now shown 
here). The same analysis, with the same conclusions can 
be done for the Moore lattice. 

The above results might shed some light on the sta- 
bility of cooperation in society. Real social networks of- 
ten have distinct communities, where individuals in one 
groups is densely connected, having relatively few con- 
nections to other groups [46[ . The emergence of commu- 
nity structure may hinder the spread of defection, just as 
in the case of the five-site cluster on the kagome lattice 
above. In this sense, our society may not be as sensitive 
to defection as indicated by the PD on the von Neumann 
lattice. On the other hand, we also show that VMLD 
is more efficient in maintaining and promoting cooper- 
ation than IPLD. Therefore, in the social situations, if 
everyone tries learn more from others, it would perhaps 
be easier for people to work together efficiently 



IV. CONCLUSION 

In summary, we have studied the effects of the direc- 
tion of strategy migration and noise on the evolution 
of cooperation in the spatial prisoner's dilemma game. 
To this end, we have considered two types of strategy 
updating dynamics, namely voter-modcl-likc dynamics 
and invasion-proccss-like dynamics, on three types of lat- 
tice structures — square lattice with von Neumann and 
Moore neighborhoods, and kagome lattice. It was found 
that the VMLD, rather than the IPLD, better sustain 
and promote cooperation on all the tested lattices. Fur- 
thermore, we found noise to have a nontrivial effect on the 
evolution of cooperation in spatial population. First, for 
low temptation to defect, coherence resonance is found 
for all three types of lattice, i.e., there is an optimal noise 
level for promoting cooperation. Second, for an interme- 
diate temptation to defect, we observe a two-peak behav- 
ior (for Moore and kagome lattices) — the first maxima 
at zero noise, the other at a moderate noise levels (on 
the von Neumann lattice, however, we observe no such 
behavior). Third, for even higher temptation levels, only 
no noise maximizes cooperation. Moreover, we find that 
in the low-noise limit, the local lattice structure deter- 
mines the fate of cooperators. We find, in the case of 
high temptation values, a structure of triangles overlap- 
ping at one site to benefit cooperation compared to a 
structure of triangles overlapping at multiple sites. For 
low temptation values, the situation is reversed and the 
multiply overlapping triangles arc beneficial. Our MC- 
simulation results are in good agreement with theoretical 
predictions obtained from mean-field cluster approxima- 
tion methods. These results may enrich our knowledge of 
the evolution of cooperation in spatially structured pop- 
ulations. A possible extension of the present work is to 
consider evolutionary PD on degree-heterogeneous net- 
works with the incorporation of asymmetric direction of 
strategy migration. We expect that degree-heterogeneity 
would induce a stronger difference in the final results, as 
was found in voter model on such networks [35l . [47j . 
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APPENDIX A: A CONCRETE EXAMPLE 

In this Appendix we present a small example illus- 
trating how the voter-like dynamics supports cooperation 
better than the invasion-like dynamics. For the sake of 
simplicity we consider a small system — a 4 x 4 square 
lattice with von Neumann neighborhood and periodic 
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FIG. 9: Illustration of a small system with 4x4 players. 

(a) Both VMLD and IPLD give out the same transformation 
probability for j to change from a defector to a cooperator. 

(b) The VMLD favors better the diffusion of cooperation than 
the IPLD when the interface is rugged. 



Thus, on a smooth interface, both VMLD and IPLD 
will result in the same probability for strategy-migration. 
Since the defectors on the interface gain much less pay- 
off than their cooperative neighbors (b is just little larger 
than 1), we can expect cooperators could invade some 
of them as time evolves. Now, assume the configuration 
in Fig. [HJb), where the player j is surrounded by two 
cooperators, i and k. In such case, if j is selected to up- 
date strategy and VMLD is used, the probability for the 
player to become a cooperator is 



W D - 



1 



1 



1/ 

1\ W H 



exp[(26-3)/K 

\-Wkj) 



} 1 + exp[(26 - l)/« 
(A2) 



where Wij/A and denote, respectively, the probabil- 

ity of i's strategy, fc's strategy convincing j. If, however, 
IPLD is adopted, the individual j could be imposed on 
the cooperative strategy either by i or k, whose proba- 
bility is given by 



boundary conditions. Initially, half of the sites of the 
lattice are occupied by cooperators, and the other half 
by defectors as shown in Fig. [Ha). We focus our at- 
tention on the probability of strategy-migration of one 
focal player, say j, changing from D to C. In the case 
of VMLD, this may happen when the individual j is se- 
lected to update its state and choose the neighbor i as 
reference, which happens with probability 1/4. Accord- 
ing to the payoff matrix of the PD, the individual i will 
get payoff 3, since there are three cooperators and one 
defector in its neighborhood. In contrast, the individual 
j acquires only payoff 6, which comes from the interac- 
tion with i. As a consequence, the expected probability 
for the player j changing from D to C is 



W D ^c = 7 



1 



41 + exp[(&-3)/«] 



(Al) 



if VMLD is adopted. This is also the transformation 
probability if IPLD is implemented, rather than VMLD. 



TP, 



D^C 



-it 



W l j + W k j 



UjjWkj \ 

4 J ' 



(A3) 



which is smaller than the corresponding value from 
Eq. (|A2[) . Note that we have assumed the neighbors of i, 
j, and k, have not changed their strategies between the 
two events (from the updating time of i to that of j). In 
sum this illustrates how the VMLD favors the migration 
of cooperative strategy along the interface separating co- 
operators and defectors as compared to the IPLD. Note 
that this argument cannot be reversed to hold for cooper- 
ators instead of defectors (considering say i in Fig. [^a) 
being a defector). The reason is that, on average, the 
payoffs of defectors alon g th e interface are smaller than 
those of the cooperators [l9l . l33j , which disables the ben- 
efit of the VMLD, and thereby inhibits the diffusion of 
defection. 
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